-
Notifications
You must be signed in to change notification settings - Fork 68
[PLT-1008] Export content parses incorrectly #1628
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see the approach and agree with it. The code looks right, not enough deep understanding on my end to dig into each and every line.
28d672c
to
8bfe14c
Compare
d75ce34
to
705275b
Compare
Are you technically deprecating the FileConverterType with this change since the buffered stream does not look like you can pass custom converters? |
NIT: one last thing: would it make sense to change the default value for the convertor in the get_stream method to the buffered version so the deprecation messages do not fire twice/ clean up the method easily? This is for this workflow. |
Sort of. I think the way it is written is a bit incorrect (yielding a file constantly is very odd in my view) but the main idea of yielding an entire file is correct. The Export_Task code is very over engineered and buggy and needs to be simplified. We should aim to give customers basically 3 outputs: fully outputted in memory, line by line in memory, or an entire file by disk and that's it. Allowing for a lot of customization such as choosing what chunk to export at (offset) or line makes etc. just makes our life more difficult maintaining the SDK. |
No. BufferedJsonOutput has a schema of This is also why buffered_stream was introduced. |
Uh oh!
There was an error while loading. Please reload this page.